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Models of Markov processes 
with a random transition mechanism 


Y. Belopolskaya^ Y. Suhov^ ^ 

Abstract. The paper deals with a certain class of random evolutions. We develop a construction 
that yields an invariant measure for a continuous-time Markov process with random transitions. The 
approach is based on a particular way of constructing the combined process, where the generator is 
dehned as a sum of two terms: one responsible for the evolution of the environment and the second 
representing generators of processes with a given state of environment. (The two operators are not 
assumed to commute.) The presentation includes fragments of a general theory and pays a particular 
attention to several types of examples: (1) a queueing system with a random change of parameters 
(including a Jackson network and, as a special case: a single-server queue with a diffusive behavior 
of arrival and service rates), (2) a simple-exclusion model in presence of a special ‘heavy‘ particle, (3) 
a diffusion with drift-switching, and (4) a diffusion with a randomly diffusion-type varying diffusion 
coefficient (including a modihcation of the Heston random volatility model). 
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1 Introduction 

This paper presents a construction of Markovian models of random processes in a random environ¬ 
ment. The problem can be stated in the following form. Suppose we are given a family of ‘basic‘ 
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continuous-time Markov processes (MPs) on a state space X where z is a parameter describing 

a ‘state of environment‘ (SE) which can be varied within a space Z. Assume that 'i z ^ process 
has an invariant measure (IM) Further, suppose that we have an MP Z{t) on Z with an IM 
V. Is it possible to construct a ‘combined‘ (two-component) MP {Z{t),X{t)) on 2xX in a ‘universah 
manner, allowing a natural interpretation of an MP in a random environment? In this paper we put 
forward such a construction (under certain limitations); a feature of our constructions is a mutual 
impact of an SE and a state of a basic process upon each other. In other words, we describe a general 
mechanism where the state of a basic process influences a change of an environment which in turn 
leads to a new transition rule for the process. Such an approach should be compared with a body 
of work on random walks in random environment (cf., e.g., [20]) where an environment is randomly 
chosen but kept hxed throughout the time dynamics. 

An example of a system to which our construction can be applied is a Jackson network (JN) 
model; see 0. HI- Here the environment can be identihed with a triple (A, p., P) where A and p are 
vectors of arrival and service rates, and P is a routing matrix. The IM is a product of geometric 
distributions identihed in terms of A, p and P. Making triple (A, p, P) dependent on parameter z 
varying randomly within a hnite or countable set Z leads to a number of interesting applications, 
viz., networks with ‘distinguished customers‘; cf. [1] and na. Our construction for this example is 
carried in Section |21 it leads to a ‘modihed‘ product-form IM for the combined MP and - after the 
normalization - to a combined equilibrium probability distribution (EPD). 

In Section [3] we discuss another instructive example: a simple exclusion model on a cubic lattice 
interacting with a special ‘heavy‘ particle. Here again, the poduct-form for an EPD of a combined 
MP is preserved in the course of the cnstruction. 

In Section H] we turn to general continuous-time MPs. First, we cover a general class of jump 
MPs. Then a general form of our construction is provided, explaining the main mechanism behind 
it. In Sections [5] and [6] we focus upon examples where one or both components are diffusions. This 
includes queues where parameters exhibit a diffusion-type behavior as well as examples of a diffusion 
with a jump-like change of the drift coefficient. In particular, in Section El we discuss several models 
related to mathematical hnances: here basic MPs are Ornstein-Uhlenbeck diffusion processes with 
varying volatility. 

A number of results in this paper are stated in the form of weak invariance equations (WIEs) 
involving a measure and an operator acting on functions of a combined state (in a simplihed form - 
a collection of combined jump rates). In cases where we are able to assert the existence/uniqueness 
of a combined MP, the WIE implies a genuine character of an IM; in these cases we refer to the 
invariance property directly. For convenience, the WIE is stated anew in each specihc (or general) 
context it is used. It has to be stressed again that our results depend on a special choice of the 
transition mechanism (inherited from Ref. |1]) where the SE and the basic MPs have a particular 
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influence on each other. 

Throughout the paper, we repeatedly quote books 0.111]. ra and [13] for various general results 
on MPs and their generators. In fact, our models can be considered as examples of random evolution 
models considered in [3], Chapter 12, although we focus here on different aspects of behavior. Also, in 
contrast to [3] , we - as was said above - include a mutual impact of the environment and basic MPs 
upon each other, albeit in a rather specihc form. (To make a comparison: the corresponding terms 
used in [3] are driving and driven processes.) Technically, in the provided assertions we attempt to 
stick to minimal assumptions (as a rule, mentioning them in passing). But a mathematically minded 
reader should pay attention to remarks where we build links with more advanced conclusions by 
using general results from 0. n. ra and [T3|. 

2 A Jackson network in a random environment 

2. A. Open Jackson networks. In this section we use a Jackson job-shop network as a background 
model of a basic MP. The JN model is dehned by the following ingredients [a do]. 

(a) A hnite collection A of sites; a hrst-come-hrst-served inhnite-buffer single-server queue at each 

site j G A. 

(b) Two vectors: A = (Aj), A* > 0 being a Poisson arrival intensity at site i G A, and z/ = (/Xfc), 

Pfc > 0 being the service intensity at site /c G A. Arrivals at different sites are independent. 

(c) A sub-stochastic matrix P = (pj^, i, /c G A): pj^ > 0, — 1- After completing service at 

site h a task is transferred to site k with probability pj^ and leaves the network with probability 
P? = 1-EfcgAPifc- 

The above description gives rise to a continuous-time MP with states n = {ui, i G A) G 
where n* G := {0,1,2,...}; this will be a prototype of the basic MP The 

generator matrix Q = {Q{n,n!)) of the process has non-zero entries corresponding to the 
following transitions: 

Q{n,n + e^) = A, an arrival of a task at site i, 

Q{n,n — e*) = /ijp*l(ni >1) a task exits from site i out of the network, (2.1) 

Q(n, n-f e*^^) = /iiPjfcl(nj >1) a task jumps from site i to k. 

Here e* = (ej, / G A) G has e\ = du, — e\ and 1 stands for the indicator. 

(d) Let I denote the unit matrix. The total intensities pi of the flows through sites i G A form a 


3 


vector p = (yOj) where 

p = A(l — P)"\ where I — P is supposed to be invertible. (2.2) 

(e) The IM u for the process is given by i/(n) = Higa iPi!i TL = iP'i)- 

The sub-criticality condition (SCC) reads: pi/Pi < 1, z G A, and gives rise to an EPD tt in the form 
of the product of geometric marginals with parameter Pi/pp 

= [riiGA {pi/PiT] /S, n = (ui), where H = EnszA 7r(n) = HiGA (1 “ Pi!Pi)- (2-3) 

In fact, assuming that matrix P is irreducible (i.e., P^ has strictly positive entries for some positive 
integer s) and that the SCC holds, the JN MP is positive recurrent, and tt is the unique EPD. 


2.B. The combined generator. We now give the description of the model modifying the 
basic example from |1]. Let ^ be a finite or a countable set. The state of the combined MP is a 
pair [z^n) where z ^ Z indicates a state of the environment (SE) and n G a state of a basic 
process. Furthermore, given 2 ; G .Z, we £x a collection of triples P(^)), with A(^) = (Af^), 

p^^^ = P*-^^ = (p-fc^), assuming that, z E Z, Eqns fl 2 . 1 l) - fl 2 . 2 l) hold, and matrix pC) is 

irreducible. The jump rates form an (inhnite) generator matrix R = (^R[{z, n), {z', • The entries 

R[{z,n), {z',n/)] are: 


+ e*)] = a{z)\^^^ 

R[{z,n),{z,n- e^)] = ^p*h)i(ni > 1 ) 

iz,n + e'-^^)] = a{z)pl''^pl/}l{ni > 1 ) 
R[iz,n), (P,n)] = a{z)T^^'>{z,z') H 
R[{z,n), {z/n/)] = 0 


R[{z,n), ( 2 ;,n)] 


E R[{z,n),{z',n/)]. 

{z' ,n')eZxZ^\{{z,n)} 


a task arrival at site i under SE 2 ;, 
a task exit from site i under SE z, 
a task moves i ^ k under SE z, 

the SE changes from z to z', 

V other pair (z', rP) E Z x TJ/_ 
with {z,n) 7 ^ iz/n/), 


(2.4) 


Here ( 0 ( 2 ), z E Z) E and {cr{z), z E Z) E Mf are given vectors with entries a{z) G [0,cxo), 
a{z) E (0,oo); they represent two forms of time-scaling: for the basic MP under SE z and for the 
exit from 2 ;. Next, T*^-) = z')) is the nominal SE-jump intensity matrix (possibly, depending 

on n) for which we assume that 


s') > 0 , 2), t(z,z)=0, V z<zZ,n<zZ‘ 


(2.5) 


One can see that the top three lines in fl2.4p emerge from the jump rates of a basic MP Xh)(t) 
whereas the bottom line is related to the SE MP Z{t). 
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We say that measure rj on Z x and the collection of rates R = {R[{z,n), satisfy the 

WIE (the weak invariance equation) if, W {z,n) ^ Z x 


E il{z,n)R[{z,n),{z',n/)] = E v{z',iTf)R[{z',r]/), {z,n)]. 

{z' ,n')^{z,n) {z' ,n')^{z,n) 


( 2 . 6 ) 


Theorem 2.1. (I) Set: 

= friiGA fpfV/^P 


jcy[z), z & Z, n= (rii) E Z^. 


(2.7) 


( 2 . 8 ) 


Then measure n and the collection of rates from Eqn fl2.4p satisfy the WIE. 

(II) Assuming the SCC 

pfVpP <l y i,z and H = H^ga (l “ j < oo 

yields a PM tt satisfying the WIE: 7r(z,n) = Higa /[“(^)'^('2^)]- 

Proof. Assertion (II) follows from (I) so we focus on the proof of (I). We check partial balance 
equations: V {z^n), we assert that F°'^*(z,n) = F‘"( 2 ;,n). Here 

F°"t(z,n) = E(^>')'^(aa)^[(aa),(^',E)], E“(2 ,n) = Ep',n') a'). (^.Zf)]- 

It is convenient to represent F°"*( 2 ;, n) = F°"*( 2 ;, n) + F‘°( 2 ;, n) = F™{z,r^ + F™{z,ri), 

and prove that Ff^^^z^rd} = Ff^{z,n), Ff'^^{z,n) = Fffz,n), with 


= Y.n' '^i^^E)R[{z,n),{z,n^)], Ff'^^iz,]].) = Y.z''^i^^E)R[{z,n), {Z,n)], 


and 


F“(z,n) = En'«^(^>A')^[(AE),(AA)], Fl^{z,n) = J2z'^E)R[iz',n), {z,n)]. 

After omitting the factor 1 /(t(z), the equation Ff'^^{z,n) = Ff^{z,n) means that 

En' = En/ iyy\rf)Q^^\rd , r) 

which holds as fd) is an IM for Q^f Next, Ff'^^{z,n) = Ff^{z,n) is equivalent to fl2.5p since 

pf I pT) * TdP{z, z') Yli [p?/p^i^) ' (^{z) = E,/ rZ\z, z') 

and similarly F^{z,r^ = E^/ t^-\z\ z). □ 

Remarks. 2.1. Note that values a{z) and ZF)(^z,z') do not enter expression fl2.7p . However, if 
a{z) = 0 then V fixed n pairs {z,n), z E Z, form a closed communicating class supporting an IM 

f 71) 

kq = Kq with values Ko{z,^ = l/a{z). 
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2.2. Under additional conditions of a ‘moderate growth of functions a{z) and a{z) (cf. Theorem 
3.1, P. 376 and Corollary 3.2, P. 379 in [3]), the collection of rates fl2.4l) dehnes a combined MP 
{Z(t),X{t)) on Z X with a Feller semi-group of transition operators. (Here it means that, V 
t > 0, the transition matrix of the process takes the space of bounded functions {z, n) ^ (j){z, n) to 
itself). Physically speaking, for ‘nice‘ a{z) and a{z) we obtain a non-explosive combined MP. (For a 
hnite set Z this is automatically true.) 

Formally, a sufficient condition (deduced from the aforementioned results in [3]) is that the fol¬ 
lowing three assumptions (i)-(iii) hold, (i) For R{z,n) = —R[{z,n), {z^v^] we assume: 


sup R{z,n) < oo where 

(z,n)g.ExZ^ 


R{z,n) = a{z) ^ rjp H ( ^ 1 + ^ • 


h)' 


h) I ,,(d- 


(2.9) 


z'gZ ieA \hi 


ieA 


Next, (ii) after compactifying space Z x by a point A, we suppose that V hnite subset JC E Zx Z^, 


I^[(ua),(2:',a')]| = 0 , 

(z,n)^A ^^ ' ' 


( 2 . 10 ) 


(z',n')e/C 


Finally, (hi) we assume that 


sup ^ 

(z,n)eZxZ{l 


z, n) — R{z', n') > R[{z, n), [z\ n)] 


< oo. 


( 2 . 11 ) 


Results about stationary distributions (see Proposition 9.2, P. 239, from [3]) imply that under 
assumptions (i)-(iii), a measure satisfying the WIE is invariant under the transition semi-group. 
Furthermore, under assumptions (i) - (hi) and condition fl2.8p . the combined MP {Z{t),X{t)) is 
positive recurrent. 

2.3. In assumption (i) (cf. Eqn (I2.9p ). the troublesome summand is 


R^^\z,n) := (t{z)J2^z7'YI 


{n) TT I h 


'ez ieA \ P\ 


O' 

i _ 

o 




A sufficient condition for sup Rz {z,n) < oo covering a host of realistic situations is that (iv) 

{z,n)eZxZ^ 

T^} is of the form = h{n) T^z' where Tzz' does not depend on n and has sup ^ T^z' < oo 

zez z'&z 


and (v) h{n) is such that sup h{n) 


R 


z£Z,n€^Z^ ieA V P' 


O' 

i _ 

(^) 


< cxo. For example, assuming that S{n) : = 


supJJ^ ( ^ value h{n) can be selected as S{n) 


O' 


-1 


ieA \Pi 
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3 A simple exclusion model in a random environment 


In this section, we focus on a class of MPs with local interaction arising from a symmetric simple 
exclusion model; cf. ra-ia. The state x of the basic MP is considered as a spin-0, 1 configuration 
(aka an occupancy conhguration) on a cubic lattice if, i.e., as a function i G i—)■ Xj G {0, 1}. 

The value x* = 1 means that site i is occupied by a ‘particle‘ whereas Xj = 0 means that site i is 
vacant/empty. (We also will use the term a ‘light particle‘ as opposite to a ‘heavy‘ particle introduced 
below.) Let us fix parameters 

ip G IP, A, /i ^ 0, l^i'i 

where i,i' G (in = 0 and sup ^ (5^/ < oo. 

i’GZd- 


Next, we take X = {0,1}^'^ with the product topology (and - when necessary - with a metric 
generating this topology), and set Z = T where T C is a finite set. Further, for x = (xj, i G Z'^) G 
{0, and 2 ; G T we write: 

lS^)g{x) = Y. xi(l - Xi/)/3iy [g{x + e*^*') - g{x)\ \ 

+A [g{x + e*) - g{x)\ + x^p [g{x - e^) - g{x)\ | 

TAe"^ [g{x + e^) - g{x)] + x^/i [g{x - e^) - g{x)]. 

Here e* G {0, is the configuration where all values of spin are 0 except for that at site i, which 
is 1, and we set: := e* — e*/ (This covers the case where i or i' coincides with z.) Adding and 

subtracting configurations means addition and subtraction of functions. 

A construction below repeats that from [13], Sect. 1.3. Denote by Cl({ 0, 1}^“^) the space of 
continuous functions C({0,1} x Z‘^). Operator in fl3.2l) acts initially on the space Cl({ 0, of 
Lipschitz-type functions x G {0,1}^“^ 1 -^ g{x)- 


CL({ 0 .iy) = |9eC({0.iy'): |||g|||:= 

+ Y - ^i) \9 {x + e^) - g{x)\ + Y 


sup 


^ Xi \g{x-e^) -g{x)\ 


Xi(l - Xi/) \g{x + e^^^') -5'(a;)| 


< 00 


(3,3) 


Next, we extend fl3.2p to a closed operator in C({0, l}^"^). The closed operator is still denoted by 
for its domain we use the notation D(L^‘^^). By construction, Cl({ 0, l}^'*) is a core for Physically, 
is a generator of an MP on the state space {0,1}^"^ representing an ‘open‘ simple-exclusion 
model, in presence of a (single) ‘heavy particle‘ placed at site z ^ T. In this model, a light particle 
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can jump from site i to site i' at rates or depending on the status of sites i and i' (vacant, 

occupied by a light particle, occupied/not occupied by a heavy particle). In any case, jumps are 
performed only if the simple-exclusion restriction is respected: at most one light particle at any 
given site. (A simultaneous presence of a light and the heavy particle at a given site is allowed.) A 
light particle can also be annihilated at rate fi and created at rates A or Ae"^, depending on whetger 
the given site is occupied by the heavy particle. Factor e'^ indicates an impact that a heavy particle 
has on the dynamics of a light-particle conhguration x\ when > 0 , the heavy particle attracts 
the light ones, when 9 ? < 0, it repels them. Following [l3]-[T4], it is possible to check that V 2 ; G T 
there exists a unique Feller semi-group of operators in C({0, 1 }^“^) generated by See, e.g., [T3] . 
Theorem 1.3.9, P. 27, or [H], Theorem 4.68. This yields a basic MP in {0, 

Furthermore, an IM (in fact, an EPD) for process is given by 


= 


p(d 


X Q 


h) 


(3.4) 


It is a product-measure (aka an inhomogeneous Bernoulli measure) where the spin values are inde¬ 
pendent for different sites, and the marginal distribution for an individual spin is either ~ p or 
~ Q, depending on whether i ^ z (i.e., site i not occupied by a heavy particle) 01 i = z (i.e., 
i contains the heavy particle). Both P and Q are probability distributions on the two-outcome set 
{ 0 , 1 }: 


P(l) = 


A 


-, P(0) = 


^ -, Q(l) = , , Q(0) = 




(3.5) 


X -\- fi A -p /i Ae*^ -p fi Ae*^ -P p 

All measures are absolutely continuous relative to 7 = x ph), with the Radon-Nikodym den- 

ieZ'* 

sities 


m{z, x) = 


du^^^x) A-P/i ■ 
d7(a:) Ae^ -P /i 


e‘^l(x 2 = 1 ) -p l(x^ = 0 ) 


A -p /i 


3 X 2 P 


Ae*^ + /i 

where ^ G T and x = (x*) G {0, 


(3.6) 


We also have that L^'^^g(x)diy^^^x) = 0, V z E T and g G Cl({ 0, l}^"^). 

Next, the SE (state of environment) process Z{t) has the generator A dehned by a hnite matrix 
for a function/vector z eT i-E- f{z), 


A/(^) = E Tzz'[f(,z') - f{z)] where rates obey 

z'eT 

Tzz' = T^'z > 0, and Tz:z = 0. 


(3.7) 


Process Z{t) is a random walk in T with a counting invariant measure 7 : 7 (D) = 7 )^©, D C T. The 
invariance property simply means that ^ r^z' = 'Yh '^z>z and is deduced from the symmetry condition 

z' z' 

Tzz' = Tz'z in 03.71) . It implies that Y = 0 nny function /. 












(3.8) 


Further, consider a measure KonTx{0,l}^‘*: 


k{z, A) = 


a{z) 


z eT, A C {0, ly 


The combined generator R is constructed from the action 

R4>{z,x) = a(z)L''V(z,i) + ^y^A4>{z,x). 


(3.9) 


Here a{z) > 0 and a{z) > 0 are time-scaling coefficients. Next, acts on the section map 
X E {0, (j){z,x): 


XI Xj(l - Xi/)/3jy 0(2;,x + - 0(z,a:) 

+ X |xj(l - X2)/3i^e‘^[0(z,a: + - 0(z,a: 

i&A\ i^z 

+\ [0(x, X + e*) - 0 ( 2 , x)] Xifi [Az, X - e*) - (j){z, x)] | 
-fAe*^ [4>{z, X + ey — (j){z, x)] -f x^/i [(j){z, x — e^) — 0(x, x)]. 


(3.10) 


Further, 


(The constant factor 


Xe'^ + /i 


A(p{z,x) = ^ r22/[0(z',x) -4>{z,x)]. 


has been absorbed into cr(x).) 


(3.11) 


X fi 

Let C(T X {0,= C({0,be the space of continuous functions Cl(T x {0,1}^^*) on 
T X {0,As in the case of we consider R initially on space Cl(T x {0,1}^“^) of Lipschitz- 
type functions: 


(3.12) 


Cl(T X {0,1X“) = e C({0,1X“) : V z e T, 

the section map x G {0,1}^“^ i-)- 0(2, x) lies in Cl({ 0, l}^"*); cf. 03.31) 


Then take the closure in C(T x {0,1}^”^), keeping for the obtained closed operator the same notation 
R. By construction, Cl(T x {0,1}^'*) is a core for R. Owing to Theorem 3.9 from [13] and/or 
Theorem 4.68 from la, there exists a unique Feller semi-group on C(T x {0, l}^"^) generated by 
R; the corresponding MP is denoted by {Z(t),X{t)). The trajectories of process {Z(t),X{t)) are 
right-continuous maps [0, cxo) t—)■ T x {0,1}^"* with left limits. 


Theorem 3.1. Under the above conditions, n is a finite IM for process {Z{t),X{t)). 
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Proof. By virtue of Proposition 1 . 6.10 in [13], P. 52, it suffices to check the equation 
/.|Px{o i}zd (R. 0 )( 2 , a:)dK( 2 ;, x) = 0 for 0 G Cl(T x { 0 , But for any such function 0 , 

/txIopF-" x) 

= E ^^/{oi}Z‘^(L^^V)(^,a;)e^^^d7(x) + L ^ ^ (A0)(2;, x)d7(x). 
Observe that V 2 ; G T, owing to the fact that the section map of (j) belongs to Cl({0, 1 }^"*), 
[ (L(^V)(^,3:)e^-^d7(x)= [ (L^^^(p){z,x)du^^\x) = 0. 

7{0,i}zd Xe‘^ + p ijopp'* 

Also, for 7 -a.a. X G { 0 , we have that Y1 = 0- Hence, (R 0 )dK; = 0 . 

2ST 


4 A general construction 

4. A. Jump Markov processes. Theorem 1 admits an extention where {Z, v) is a (standard) mea¬ 
sure space. (Usual measurability and countable additivity assumptions must be adopted throughout 
this sub-section.) Here, sums J2z E 2 ' replaced by integrals against measure v; measures on 
space Z X are supposed to be given via (Radon-Nikodym) densities wrt v times the counting 
measure. Viz. in (12.Sh one reads H = | HisA /'^('^)| dv{z) < 00 . Existence of 

the combined MP (Z(t), A(t)) should be analysed separately; cf. Remarcs 2B, 2C. 

Next, can also be replaced with a measure space, (X, 7 ); here the combined state space is 
again the Cartesian product (^ x X, n x 7 ). We refer the reader to in [3], Section 2 of Chapter 4, 
PP. 162-173, and Section 3 of Chapter 8 , PP. 376-382 , for a detailed treatment of jump MPs and 
their generators. Assume that for x G X we have a kernel t^^\z., z') > 0, z, z' G and for z ^ Z o. 
kernel Q^^)(x, x') > 0 and a function m{z,x) > 0, x,x' G X, such that t^^'^{z,z) = Q^^\x,x) = 0 and 


z')dv{z') = J^T^^\z',z)dv{z'), 
m{z,x) J^Q^^\x,y)d-f{y) = J^m{z,y)Q^^\y,x)d-f{y), 


for {v X 7 )-a.a {z,x) G Z x X. (4.1) 


Physically speaking, Eqn (14.R yields that the environment MPs have an IM v whereas the basic MP 

(dx) 

under SE (state of environment) 2 has an IM with Radon-Nikodym density — r-r^ = m(z,x). 

7 (dx) 

(As above, a formal construction of these MPs requires additional assumptions discussed in [3].) 
The combined generator has rates R[{z,x), (V,x')]: 


R[(z,x), {z,x')] = a(z)Q^^\x,x'), Pl(z,x), (z',x)j = 


(4.2) 
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with time-scale coefficients a{z) > 0, a{z) > 0 having the same meaning as before. We say that a 
density r}{z, x) on (Z x X, n x 7 ) and kernel R = {R[{z, x), {z', x')]} satisfy the WIE if, for (n x 7 )-a.a 
{z, x) & Z X X, 

'niz,x) J 2 ^^R[{z,x){z',x')] 1 {{z,x) 7^ [z',x'))dv{z')d-i{x') 

= ^ {z',x'))dv{z')d'^{x'). 

Theorem 4.1. (I) The density k{z,x) = u^^\x)/a{z) considered relative to measure n x 7 
and the kernel R[{z,x),{z\x')\ from Eqn fl4.2p satisfy the WIE on Z x X. (II) Under the SCO 
H = f^^^K(z, x)dv(z)d'y(x) < 00 , the PD 7r(z,x) = k{z,x)/S also satisfies the WIE. 

Proof. Repeats that of Theorem 2.1 by replacing snms with integrals. □ 

Remarks. 4.1. As before, the kernels t^^\z, z') enter the density n indirectly, throngh measnre 
V and condition fl4.ip . 

4.2. Under additional assnmptions on spaces {Z,v) and (X, 7 ), functions a{z) and a{z) and 
kernels z') and QR^x., x') (see Theorem 3.1, P. 377 in [3]), operator R generates a strongly con¬ 

tinuous Feller semi-group on ^ x X. It can also be achieved that k yields an IM for this corresponding 
MR 


4.B. General assumptions on component MPs. In sub-sections 4.B - 4.D we develop 
further a general construction of a combined generator. Until the end of Section 01 the spaces Z and 
X below are supposed to be locally compact Polish, and all measures under consideration are Borel, 
countably-additive and hnite on compact sets. 

Let us proceed with formal definitions. As before, an SE is represented by a point z ^ Z. We 
also assume that we have a measure v on Z which will serve as an IM for environment MPs. 

Next, there is a space X given; points x G X are treated as states of a family of MPs indexed 
hy z & Z - we again call them basic MPs. Physically speaking, each basic MP has a generator and 
an IM indexed hy z E Z. More precisely, V z G 2, we have a measure on X; we suppose that 

each measure is absolutely continuous wrt a fixed measure 7 , with Radon-Nikodym densities 
dv^^\x) 


m{z, x) = 


d7(a;) 


> 0 for {v X 7 )-a.a. {z,x) E Z x X. 


Further, \/ z E Z a closed linear operator is given, acting on functions g : X ^ M. forming a 
domain D(L’^^^) dense in Cb(X), the space of bounded continuous functions on X. Further still, we 
suppose that each is a generator of a Feller MP on X. That is, is a conservative operator 
satisfying the assumptions of Theorem 2.2, P. 165 in [3] (a version of the Hille-Yosida theorem). 
We also assume that V g from a core of the integral Jjj(L(^) 5 f)(a;)dz/(^)(a:) exists and equals 0. 
(Formally, it is only the latter assumption that matters in the proof of Theorem 4.2 in the next 
sub-section.) 
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Likewise, we suppose that V x G X there is a closed linear operator acting on functions 
f : Z ^ M. forming a domain D(A^^)) dense in Cb(-Z), the space of bounded continuous functions on 
Z. In addition, we assume that each A*^*^ is a generator of an Feller MP on Z. We also suppose that 

V / from a core of the integral f^(A^^^f)(z)dv(z) exists and is equal to 0. (Note that measure 

V serves all operators A^*b) 

4.C. A combined generator. We are now going to introduce generator R and analyse the 
related WIE. Fix a time-scale function z G Z a(z) with cr(z) > 0 for u-a.a. z ^ Z. Consider the 
following measure k on 2 x X: 

K(i?) = [ m{z,x)d'y{x)dv{z), V Borel B C Z xX. (4.4) 

J 

Then introduce a linear map R acting on functions {z, x) E (Z x X) (j){z, x) G M, 0 G D(R), where 

R0(2;, x) = A^^'>(j){z, x) -I- 3;). (4.5) 

Here, it is understood that A^^'>(p{z, x) results in the action of A^^^ in SE-variable z succeeded by 

multiplication by l/m(z,x), and L*^^^0(^,x) results in acting in variable x G X. Formally, for 
(t> = f with 0 ( 2 r, x) = f{z)g{x): 

|A<-’'(/0 9 )](^.i) = 0g)]{z,x) = a{z)f(z)(Li‘>g)(x) (4.6) 

TT}/ {Z>)X] 

where z E Z a{z) > 0 is another given time-scale function. Referring to a Banach space 
Cb(^)<8)Cb(X), with a chosen cross-norm, the action of R is then extended by linearity and continuity 
to general functions 0. 

In this paper we do not intend to go into details of formal constructions of a closed operator 

based on map R defined in fl4.5p . Nor shall we try to analyse the conditions of the Hille-Yosida 

theorem for such an operator. (Still, we will refer to this operator as R and suppose that it exists.) 
It will be assumeed that the domain D(R) contains functions 0 G Cb{Z) Z Cb(X) such that (i) V 
z E Z, function g^^z ■ x i—)■ (j){z,x) belongs to D(l(^)) (ii) V x G X, function ■ z i-G- (J){z,x) 
belongs to D(A*^*)). (We will say that functions fip^x and gip^z give section maps generated by function 
0.) Pictorially, we would like to treat R as a generator of an MP {Z{t),X{t)) on Z x X which is a 
superposition of two tendencies: one is to keep an SE value z intact and evolve in component X in 
accordance with the MP generated by a{z)L^^\ the other to keep state x of the base MP but change 
the SE value z by following the MP on Z generated by a{z)k^^\ (As examples in this paper show, 
under additional assumptions such a process can be constructed.) 

For functions 0 satisfying (i), (ii), formula fl4.5p becomes 

R0(z,x) = [A(^V<^.x](^) + [L(^i^<^,^](x). 
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Theorem 4.2. The following properties (I)-(II) are satisfied. (I) Measure k and operator R 
from fl4.4p - fl4.6l) obey the WIE in the sense that the integral exists and equals 0 V 

function (j) for which 


U,x]{z) dv{z) ^d 7 (a;) < oo and / f^^x]iz)dv{z) = 0 , for y-a.a. x eX, 


and 


a{z) 

a{z) 


g^^z\{x) du^^\x) I dv{z) < oo and / g^^:^{x)dv^^\x) = 0, forv-a.a. z ^ Z. 


(II) If S = k{Z xX) < oo then Tr{B) = k{B)/'E yields a probability distribution on Z xX, with 
f^^^('R(/))d7r = 0 V (/) as in the above definition. 


Proof. As before, we focus on assertion (I). Here, for a function 0, satisfying the conditions of 
the theorem, 

J {Rf))dK = J jy [k^^^ f^^^]{z)dv{z)'^ dy{x) + J |^ [h^"h 4 >,z]{x)dv^^\x)^dv{z). 

Both summands in the RHS vanish owing to the assumptions. □ 


Remarks. 4.3. As in previous results, the time-scaling factor a does not enter expression for 
the IM in fl4.4p . 

4.4. Operators enter Eqn (I4.4p via a common IM v. This fact leads to a time-homogeneous 
combined MP. 

4.5. The condition that « 7 and m{z,x) > 0 guarantees that the change of environment 
results in a ‘non-disruptive‘ continuation of the combined MP (should it exist). 

4.6. Like earlier parts of the exposition, additional assumptions are needed if we wish the 
combined MP to exist (some of these assumptions are hard to verify). Viz., conditions guaranteeing 
that a sum of two generators is a generator of a continuous contraction semi-group are listed in [3], 
Theorem 7.1, P. 37 and Corollary 7.2, P. 38. Sufficient conditions under which an operator generates 
a continuous positive contraction semigroup are given in m, Theorem 2.2, P. 165. The fact that 
a Feller semi-group (i.e., continuous positive contraction semi-group with a conservative generator) 
induces an MP with Skorokhod-type sample paths is established in [3], Theorem 2.7, P.169 and 
Corollary 2.8, P. 170. The equivalence of the WIE and the IM property is proved, under additional 
assumptions, in [3], Proposition 9.2, P. 239. See also [3], Theorem 9.17, P. 248. With regard of a sum 
of two generators, other relevant results are contained in [3], Section 10 of Chapter 4, PP. 253-261. 

4.7. We would like to point at an iterative property of our construction: once constructed, a 
combined MP may be used as a basic/environment MP to produce a combined process of a ‘higher‘ 
level. 
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4.8. The Markov property of basic processes can be replaced by weaker assumptions covering a 
broader classes of random processes with an inhnite memory. This can be a topic for future research. 


5 Processes with diffusive components 


Diffusion MPs play an important role in the modern theory as they are described in comprehensible 
terms and provide a broad spectrum of interesting properties. A particular feature of diffusion pro¬ 
cesses is that they are generated by second-order differential operators, and existence and properties 
of an MP are expressed here through properties of coefficients and boundary conditions. In this 
section we comment on examples of the above construction where basic MPs or an environment MP 
or both are represented by diffusions. 

5. A. Models with a jump basic process. In the current sub-section we discuss models where 
the basic process is a jump MP whereas the SE (state of environment) process is a diffusion. More 
precisely, we deal with simple, but not entirely trivial, examples of an isolated M/M/l/oo queue, 
with X = Z+ := {0,1,2,...}. We employ the notation traditionally used in the literature in this 
area. (So, A stands for the Poissonian arrival rate and p for the service rate; condition A < /i is 
necessary and sufficient for the queue to be stable.) 

5.A.I. First, take Z = (e, 1) where 0 < e < 1. Here the SE-point z = A follows a diffusion 
process with coefficient (3{n) > 0 and with reffections at the borders e and 1. For the basic MP we 
have jump rates corresponding with /i = 1; 

(5^'^^(n, n-|-1) = A, Q^^\n,n — 1) = l{n > 1), n G Z+. 


In other words, we hx the service rate p = 1 and vary the arrival rate A between e and 1 as prescribed 
by the above diffusion. The IM v on (e, 1) is Lebesgue: dz;(A) = dA, while on Z+ the IM z/A) is 
geometric, with = A”. (For the sake of convenience, we omit the normalizing factor.) The 

combined state space is (0,1) x Z+, and the combined generator R acts by 


R())(A,n) = 


a(A)/l(n)2 


0(A,n) 


2A” dA2 

-ha(A)|A[(/)(A,?7, 1) - (/>(A,n)] [0(A,n - 1) - 0(A,n)] l(n > 1)|. 


(5.1) 


The domain D(R) consists of functions (A,n) i-G- 0(A,n) that are in variable A G (e, 1) and 

d d 

satisfy the Neumann boundary condition —(/)(e-f,n) = —0(1—,n) = 0. Assuming that (i) (t(A) 

OA 0\ 

is continuous and strictly positive on [e, 1] and (ii) ^ /5(n)^/e" < cxo, it is possible to check that 

operator R from Eqn fl5.ip generates a Feller semi-group MP on Cb([e, 1] x Z+); cf. [8], Section 2.1, 
Chapter 2, or |T2], Sections 3 and 4, Chapter 15. The IMs for the corresponding MP on {Z{t),X{t)) 
with generator (15.ip are analyzed in Theorem 5.1 below. 
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Theorem 5.1. Suppose that conditions (i) and (ii) are fulhlled. (I) An IM on (0,1) x Z_|_ has the 


A" 


1 A^ 


-dA < oo (the SCC), the normalized 


form K{A,n) = J^——dX, A C (e,l). (II) IfE := f, ..y 
o-(A) n>0 o-(A) 

measure 7r(A,n) = K(A,n)/E yields an EPD, and process {Z{t),X{t)) is positive recurrent in the 
sense that V n G and interval Ac (e, 1) the mean return time to the set A x {n} is finite. 

Proof. As above, (II) is a technical corollary, and we focus on assertion (I). Here our task 

is to check that measure k is annihilated by the conjugate R*. The corresponding calculation 

K(dA, n) 


is straightforward: the shortest proof is to pass to the density 


A^ 


and check that 

cr(A) dA 

\n \n 

, , (R/)(A, n)dA = 0 V / G D(R) for which the sum ^ I (R/)(A, n) [dA < oo. 

1 - e'" 

E.g., with cr(A) = 1 (no time change in the environment MP), we have S = ^ -= oo, and 


□ 


assertion (II) does not provide an EPD. However, for cr(A) = 
A ~ 1), ^ 

S = A"(l - A)dA = 5^ 


1 


1-A 


n>l ^ 

(an acceleration/chaotization for 


1 


,n+l 


n + 1 


n>0 n>l 

and the construction in assertion (H) guarantees existence of an EPD. 


< oo. 


5. A.2. Our next example is a dual of the previous one: now we hx a value for the arrival rate 

A = 1 and let SE z = p. follow a Brownian motion on Z = (1, oo), with a drift b and a reflection at 

the leftmost point 1. The IM dv{p) on (1, oo) is (which is just Lebesgue when b = 0). The 

0 

combined generator R acts on C^-functions (/i,n) G (1, oo) x Z. t—)■ with —0(l+,n) = 0: 

op 


R0(p, n) = p^a(p) 
-\-o. 


1 d 


li,n+l)- +pcf>(iJ,n- 1) - i^(;i,n)]l(n > 1)|. 


(5.2) 


In this setting, we leave open the question of existence of the combined MP, focusing instead on the 
weak invariance equation (WIE). We say that a function {p,n) G (1, oo) x Z+ i-G- ri{p,n) (with real 
values ri{p,n) G M) satishes the WIE with generator R from fl5.2p if, V {p,n) G (l,oo) x Z_|_, the 
following properties hold: (i) 


1 <92 r 


2dp^ 


p^a{p)ri{p,n) 


•i 


p^a{p)r]{p,n) 


= 0 , 


and (ii) 


T]{p, n) [1 + pl{n > 1)] = T]{p, n - l)l{n > 1) + pr]{p, n + 1). 
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Theorem 5.2. (I) The function (/u,,n) G (l,cxo) x i—)■ of the form = 


satisfies the WIE with generator R from fl5.2p . (II) Under the SCO S := X]n>o /i”* 
K yields a probability density function. 


g2(M-l)b 


g2(M-l)b 


dfi < oo, 


The proof is done by a direct substitution and is omitted. 
Again, with a(/i) = 1, the partition function S = 


fie 




-dfi = oo, regardless of the value b. 


^ /i- 1 

To obtain a PDF in assertion (II), we need to introduce a time-scale cr(/i) growing at /x ~ 1 and - 
when b > 0 (i.e., no the drift towards 1) - at /i ~ oo. 

5.A.3. Now consider the case where the SE is a pair (A,/i) varies according to a joint diffusion 
in a space Z identihed as a vr/d-angle A = {(A,p) : fi > X > 0}, with drift ^ ^ ^, reflected at the 
sides A = 0 and X = fi, along the inward normal directions. It means that generator = A acts by 

(5.3) 




The domain of A consists of C^-functions (A,/i) G A t-G- /(A,p) satisfying the boundary conditions 

d f d d\ 

—/(0-|-,p) = 0 and 7 -— — fiX,fi) = 0. Here the IM v has the Lebesgue density 
dX \dX dfi J x=ii 


dv{X,fi) 

dXdfi 


= exp[29{X + fi)], (A,p)gA. 


(5.4) 


(A hnite IM arises iff 6 * < 0.) 

Accordingly, the generator R has the form 


R0(A , fi, n) 

^ h''tT(A,/i) 

A” 




(5.5) 


-ha(A,/i)| [0(A,yn, n + 1) - 0(A,/i, n)] /i[0(A,/i,n - 1) - 0(A, p, n)] l(n > 1)| 


and acts on functions (A,/i, n) G A x Z+ 1-4 0(A,p,n) which are and satisfy the above boundary 
conditions in variables A, fi. Repeating the previous dehntion mutatis mutandis yields 

Theorem 5.3. (I) Consider the function (X,fi,n) G A x Z+ 1-4 K{X,fi,n) of the form 

A”exp[20(A + /i)] 

i^{Xfi,n) = -—-—^-. 

h o-(A,/i) 

Then k satisfies the WIE with the generator R from fl5.5p . 
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(II) Assuming the SCC E := J2n>o Iz 
k(A, /i,, n) jE yields a PDF on A x 


A’^exp[2g(A + /x)] 

fi) 


dAd/i < oo, the normalized function 


Remarks. 5.1. The indicated form of IM v on 21 = A in Eqn fl5.4p can be obtained by observing 
that the SE diffusion process is a projection, upon A, of a ‘covering‘ MP living in a quadrant 
: X,fi > 0}. (The projection is (A,/i) E B (A A /i, A V /i) where A A /i = min[A,/i], 
A V p = max[A,/i].) The covering MP is a diffusion with the same drift vector (parallel to the 
bissectrice) and the normal reflections from the sides A = 0 and /i = 0. It is easy to see that 
the covering diffusion is a product of two ID diffusions, one in A and the other in /i, each on 
M+ = (0, cxd), with drift 6, reflection at the origin. A more general class of boundary conditions can 
also be considered, by following results from 0-0. IIBI. 

5.2. The methodology developed thus far in this section allows us to proceed with the case of 
a Jackson network. Here it is convenient to pass to vectors p and p and consider (individual or 
joint) diffusions in the corresponding wedge-like domains. (Matrix P can also be varied in its own 
simplex-type domain.) The resulting picture essentially looks like the one above; detailed questions 
are left for/can be a subject of future research. 


5.B. Models with a basic diffusion process. Models with basic diffusion MPs and a jump 
change of the environment are quite popular in some chapters of the theory of controlled diffusion 
processes (switching diffusions); see [T], |T6], [19]. Here we discuss a straightforward example where 
X = M. The basic MP lives in a line and has diffusion coefficient 1 and a drift 2 ; G M; the latter is 
considered as an SE. Consequently, dr;(a:) = dx, and the IM with m{z,x) = 

a; G M. Next, we take ^ = M and consider a jump environment MP with a jump measure dz'). 

Let V be an IM with j^v{dz)T^^\zAz') = v{dz'). The individual components become 

1 d^ d 

lS^)g(^x) =--^g{x) + z—g{x), m{z,x)=e^^^, 2 ;,a;GM, (5.6) 

k^^^f{z)= f T^^\z,dz')[((>{x, z') — (j){x, z)], 2 ;, xgM. (5.7) 

Jr 

Then set 


R 0 ( 2 ;, x) = a{z 


dK{z,x) = e‘^'"^v{dz)dx, 

I ,, . d 




X,Z E 


(5.8) 


+a{z)e J^T^^'>{z,dz')[(j){x, z') -(j){x,z)], 


simple example is where z, z' = ±1 with jump intensities T^^)(l, —1) = 1,1) = q{x); here 

the IM V = &(Ji-|-J_i), the sum of Dirac deltas supported at z = ±1, with equal coefficients. Another 
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example is where dz') = T^^^z + u, d{z' + u)) V m, z, z' G M (shift-invariant jump measures). 

Here v{dz) = dz is Lebesgue.j 

Theorem 5.4. (I) Operator R and measure k satisfy the WIE on M x M. (II) Under the SCO 

^2zx 

S := ^ ^ dxn(dz) < oo, we obtain a unique PD satisfying the WIE. 

Remarks. 5.3. As above, one can refer to Theorem 3.1, P. 377 in [3], and provide sufficient con¬ 
ditions upon T^^\z, dz') under which the combined MP will exist, with a Feller transition semigroup 
on M X R. 

5.4. The see S < oo requires a rapid growth of a{z) at z ~ ±oo. 


5.C. Models of a two-component diffnsion. The next class of models is where both basic 
and SE MPs are diffusions. The simplest model is where the basic MP is a d-dimensional 
time-scaled Wiener process (WP), with X = R'^, the SE space Z = R, and the SE process Z{t) is a 
standard WP on a line. Here 

U^)g{x) = 

i<i<d * X = (xi,..., Xrf) G R'^, 2; G R. (5.9) 

A/(.-) = bf{z) + !/"(.-), 

Moreover, the reference measure d 7 (x) is d-dimensional Lebesgue, dx, and the IM dv{z) is absolutely 
continuous wrt dz, one-dimensional Lebesgue: 


m{z,x) = 1 that is, du^^\x) = dx, 
dt;(2;) = e'^^^dz. 


(5.10) 


Both and v are genuine IMs for their respective MPs and Z(t). 

Next, fix coefficient functions 2 ; G R+ a(z) >0 and 2 ; G R+ a(z) >0 such that 


a and a are Hoelder functions, and c < z^a{z), a{z) < C{1 + z"^) 


(5.11) 


where c, C G (0, 00 ) are constants. The combined generator R has the form 




R0(z,x) = a(z)— ^ ■^(t>{z,x)+a 


l<i<d 


' d 1 <92 

b^y{z,x) + -.^,l,{z,x) 


, X eR,z e 


(5.12) 


and acts on C^-functions {z, x) i-7 (j){z, x). More precisely, R is a closed operator whose domain D(R) 
is the closure, in Cb(R+ x R'^), of the set of functions 0 such that R()) G Cb(R+ x R'^). Owing to a 
general result, such as Theorem 1.5 on P. 369 in [3], Section 8.1; see also Ref. m, Sections 5.2-5.4, 
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R gives rise to a unique Feller semi-group of operators described by transition densities (relative 
to dz X dx). Consequently, there exists a unique two-component diffusion process {Z{t),X{t)) in 
M_i_ X generated by R. Then, in accordance with Proposition 9.2 in [3], measure k, with 

^2bz 

K{dz X dx) = dzdx (5.13) 


yields a genuine IM for {Z(t),X{t)). As the basic IM is Lebesgue, 


^2bz 

/M+xR^dzda; = oo, and 


there is no EPD. 

The two-component diffusion can be described as a solution to a system of stochastic integro- 
differential equations (SIDEs) 


dX(t) = ^a{Z{t))Z{t)dWi{t), X(0) = e R'=', 
dZ{t) = ba{Z{t))dt + ^a{Z{t))dW 2 {t) + dLz{t), Z{0) = z° > 0, (5.14) 

Lz{t) = J^l{Z{s) = 0)dLz{s). 

Here Wi{t) is the WP in R"^ and W 2 {t) the WP in R, and Wi{t) and W 2 {t) are independent. Process 
Lz{t) is a local time spend by Z{s) at 0 by time t. {L{t) increases on a set of time points of measure 
0 .) 


Remark 5.4. SDEs fl5.14p involve the solution to the so-called Skorohod problem. Here we 
consider a WP W{t) on R and construct a BM Z{t) on (0, oo) with reflection at 0 by taking a pair 
of processes {Z{t),L{t)) such that (i) Z{t) = z^ + W{t) + L{t), t > 0, z^ > 0, (ii) Z{t) > 0, t > 0, 
and (iii) process L(t) has L(0) = 0 (starts at 0), is continuous, monotone nondecreasing and obeys 
supp (dL) C {t > 0 : Z(t) = 0} (that is, L(t) increases only at times when Z{t) is equal to zero). 
Moreover, (iv) L{t) = [—z^ — infs<i hF(s)]'''. In addition, (v) the distributions of processes Z(t) and 
|hF(f)| coincide. 

In a similar way, for Z = (0,1) the solution of the Skorohod problem is a triple of processes 
(Z(t), L{t), U{t)) such that (i) Z{t) = z^ + W (t) -|- L{t) — U{t), t > 0, 0 < z^ < 1, (ii) 0 < Z(t) < 1, 
t > 0, (iii) process L{t) has L(0) = 0, is continuous, monotone nondecreasing and obeys supp (dL) C 
{f > 0 : Z{t) = 0}, and (iv) a similar property holds for U{t) (repacing endpoint 0 with 1). 

An analogous construction works for Z = {z^,z‘^). 


6 Two-component diffusions based on an Ornstein—Uhlenbeck 
process 

In this section we consider several examples of combined two-dimensional diffusion where the basic 
MP is an Ornstein-Uhlenbeck process (OUP). Such examples may be of interest in hnancial calculus. 
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In all models that are discussed here, (a) we take a(z) = a(z) = 1, (b) the SE generator = A 
is independent of x, (c) space X is a line R or a bounded interval I = (x^,x^) C R, with Lebesgue 
measure d 7 (x) = dx, and (d) space 2 coincides with a half-line R+ = (0, oo) or the whole line R or a 
bounded interval J = (z^, z^) C R, and (e) v is an absolutely continuous measure v: dv(z) = w(z)dz. 
(We will loosely refer to 2^ as an interval.) The basic MP on X has generator where, for 

a C^-function x G X i—>■ g{x), 


L^^'^g{x) = —xg'{x) + —g”{x), with m{z,x) = e x G X, z e Z. 


( 6 . 1 ) 


More precisely, let Cb = Cb(X) denote the space of bounded continuous functions x E X ^ g{x) 
with the sup-norm. The (closed) operator acts on its domain D(l(^)) C Cb(X) which is the 
closure of the set of functions g such L^^'^g G Cb(X) and g satishes a chosen boundary condition 
at the endpoints in case X = I. The coefficient z^ is interpreted as a stochastic volatility. In all 
models under consideration, there exists a Feller semi-group generated by which is determined 
by transition probability densities. Viz., for X = R, the densities are 


Pt{x,x') = 


^\z 


exp 


{x' — xe 
z^(l — e~^*) 


x,x E 


( 6 , 2 ) 


In this case, after normalization, the IM density m{z,x) = z ^ 0, gives rise to a Gaussian 

probability distribution N(0, 2:^/2). 

Examples of the SE diffusion in this section will be a Brownian motion (BM) on R+ with a 
reflection at 0 (sub-section 6.B), an OUP on R (sub-section 6.C) and an affine Cox-Ingersoll-Ross 
diffusion on R_|_ (sub-section 6.D), as well as versions of these models on bounded intervals J. 

The existence/uniqueness of combined Feller MPs on bounded rectangles I x J will follow from 
regularity (Hoelderness and non-degeneracy) of coefficients/boundary conditions. See, Ref. [3], 
Section 8.1, Theorem 1.5 on P. 369, or Ref. Sections 5.2-5.4. Consequently, for such examples 

covered by these general results we will be able to use the term IM unreservedly. Otherwise we 
only work with generator R (see Eqn fl6.4lB and will have to employ the WIEs (weak-invariance 
equations). Cf. Eqn flb.lip below. 


6 . A. Let us hrst re-formulate our general construction in the situation where basic and environ¬ 
ment MPs are diffusions. As was said, we consider the case where the dimension of each component 
equals 1; however, the general scheme can be also developed in a multi-dimensional setting. The 
attention is on a two-component diffusion MP {Z{t),X{t)) in a Cartesian product Z xX where X 
and Z are of the types described above. Here, the Lebesgue measures dz and dx will play special 
roles. 

The coefficient functions 


X G R e-)- a(x) G R, z E Z ^ c{z) E M, z E Z C{z) > 0, {z,x) E Z x X hE- m{z, x) > 0 (6.3) 
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are supposed to be Hoelder, and Wi{t) and W 2 {t) are independent standard Wiener processes (WPs) 
in R. We also specify that, for a diffusions with accessible boundaries we use the Neumann condition. 

Accordingly, the generator R is a closed operator in Cb{Z x X); on C^-functions {z,x) E Z xX 
(f){z, x) its action is given by 


R<j){z, x) = a{x) —{z, x) + y a; 


50 C{z) 5^0 

/ N c(^) ^{z, x) + a 2 
m{z,x) I oz 2 oz^ 


(6.4) 


More precisely, the domain D(R) is the closure of C^-functions 0 for which R0 G Cb(2^ x X), and the 
Neumann boundary conditions are fulhlled on the accessible parts of the boundary. Here Z and X 
stand for the closure of Z and X and Ct,{Z x X) denotes the space of bounded continuous functions 
on 2^ X X. 

Viz., for X = R and Z = R+ = (0, cxo), assuming that 0 is an instant reflection point for Z{t), 
generator R introduced in fl6.4l) is dehned on a domain D(R) which is the closure of D°(R) where 


D0(R) = 0 e C2(R+ X R) : R0 e Cb(R+ X R), 

d(j){z, x) 


R0(O+,a:) = lim R0(2;,x), 

2 :-)-o+ dz 


= 0 V x e 


^= 0 + 


(6,5) 


As was mentioned, in some examples the existence and uniqueness of a Feller diffusion {Z{t),X{t)) 
follows from general existence/uniqueness theorems; when such known results are not applicable, the 
notation {Z{t),X{t)) and the term a combined process have only an inspirational meaning, and the 
term IM is a euphemism for a solution to the WIE in Eqn (16.111) . 


To construct an IM k for {Z(t),X(t)), we make two assumptions. First, we suppose that, V given 
z, function x i-G- m{z,x) gives the density of an IM for a diffusion MP in R with some 

boundary conditions. The generator of process is a closed operator in Cb(X) whose action 
on C^-functions x ^ g{x) is given by 

L^^''g{x) = a{x)g’{x) + —g'’{x)^ x eX. (6.6) 

More precisely, the domain D(L*^^^) is the closure of the set of C^-functions g for which L^^'^g E Cb(X) 
and the derivative g'{x) vanishes at the accessible points of the boundary dX. 

In other words, we assume that for a sufficient amount of functions g (forming a core for L^^^) we 
have that 

[ m{z, x) L^^'>g{x)dx = 0, implying L^^^*m(z, x) = 0, zEZ,xEX. (6.7) 

Jd 
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Here is the (properly defined) adjoint operator acting on Radon-Nikodyni densities (relative to 
dx). The reader familiar with the concepts of the scale and speed densities can think that 


m{z, x) oc exp 


^ / a{y)^y 


Second, we assnme that there is an SE diffusion MP Z{t) in interval Z which obeys the SDE 

dZ{t) = c(z{t))dt + [C{Z{t))Y/‘^m 2 {t), Z{0) = Zo, ( 6 . 8 ) 


and has an IM v. (As above, the SDE (I6.8jl is subject to a modification at point 0 when Z = (0, cxd) 
and 0 is an accessible boundary.) Here W 2 it) is a standard Wiener process in M, and dv{z) is assumed 
to be absolutely continuous wrt dz: dv{z) = w{z)dz. The generator A of process Z{t) is a closed 
operator acting on C^-functions z f{z) by 


A/(z) = c(z)/'(z) + ze Z; 


(6,9) 


its domain D(A) is the closure of the set of C^-functions / with A/ G Ct,{Z) such that /' = 0 at 
accessible endpoints of Z. Thus, it is assumed that for sufficiently many functions / (forming a core 
for A) we have that 

/ kf{z)w{z)dz = 0 implying k*w{z) = 0, z e B. (6.10) 

Jm 

Referring, as before, to the scale and speed densities, one can assume that 

2c{y) 


w{z) oc exp 


C{y) 


dy 


We say that a measure rj{dz x dx) on x X satisfies the WIE with generator R if 


R0(^, x)r]{dz X dx) = 0 


( 6 . 11 ) 


IZxl 


for any function (x, x) G x M (j){z, x) satisfying conditions (i)-(ii) below, (i) V z, the section 
map '■ x (j){z, x) lies in a core of operator . (ii) V x, the section map : x (j){z, x) lies 
in a core of operator A. (iii) The following integrals are finite: 


IZx^ 


d(j) 






(x,x) 


r]{dz X dx) 


and 


(90 C{z) (9^0 


r]{dz X dx). 


JzxRm{z,x) 

which allows us to use any order of integration in the summands emerging in the LHS of (I6.11j) . 
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Theorem 6.1. Assume fl6.7p and 06.101) . Then the following assertions hold true. (I) The 
measure k, with the Radon-Nikodym density 

K(dz X dx) 


v{dz) X dx 


= m{z, x) 


( 6 . 12 ) 


satisfies the WIE with generator R from Eqn 06.41) . (II) In case of an absolutely continuous measure 

V, with vidz) = wizidz we obtain — ^^- = m(z, x)w(z). (Ill) Assume that X and Z are 

dz X dx 

bounded intervals I C M and J C (e, cxo), respectively, where e > 0. Suppose that coefficient functions 
a(x), c{z), C{z), m{z,x) are bounded and Hoelder, with inf C(z) > 0 and inf m(z,x) > 0. Then 
there is a unique Feller diffusuion process with generator R in rectangle J x I, and 

measure k, is an IM for {Z{t),X{t)). 


Proof. Let us start with (I). We have to check that \R(t){z,x)\rL{dz x dx) = 0, for each 
function {z, x) 4>{z, x) mentioned in the above dehnition of the WIE. We write that 


/ x)K,{dz X dx) = 

' ZxR JZx 

1 




+ 


m{z,x) [ ' dz 

which is equal to a sum of integrals R + R where 


, .dcj) C{z)d‘^(j) 

c{z) — {z,x) + ^^ — {z,x) 


2 dz^ 


m{z, x)t;(d^)dx 


h = 


a(x)-(z,x) +-—y,x) 


m{z, x)dv{z) dx 


m{z, x) [L^^'^(j){z, ■)] (x)dx > dv{z) 

) J 


and 


h = 


,.d(t) C{z) d'^d, N 


v{dz)dx = 


U z 


[A0(-,x)] {z)v{dz) 


dx. 


In both expressions, the inner integrals vanish; in R it occurs due to 06.7p . and in R by virtue of 

(Elo]). 

Assertion (II) is straightforward, whereas (III) follows from general results (see, e.g., [3], Section 
8.1, Theorem 1.5 on P. 369). 


We now pass to examples of interest. 

6 .B. We start with an example where Z = M+ = (0,oo). Here, the environment MP Z{t) is 
represented by a BM in M_|_ with drift 5 G M and reflection at 0. Recall; given a standard WP W{t), 
there exists a solution to the Skorohod problem. That is, V Zq > 0, 3 a pair of processes Z{t), L{t) such 
that (i) Z{t) = zo + W{t) +L{t), t > 0, (ii) Z{t) > 0, t > 0, and (iii) process L{t) has L(0) = 0 (starts 
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at 0), is continuous, monotone nondecreasing and obeys supp (dL) G {t > 0 : Z(t) = 0} (that is, 
L{t) increases only at times when Z(t) is equal to zero). Moreover, (iv) L{t) = [—Zq — inf^^^ hh(s)]'''. 
In addition, (v) the distributions of processes Z{t) and |hh(t)| coincide. Formally, we have: 


r I'z'i / \ // N z‘^g"(x) . . 

^g[x) = —xg [x) H- - -, m[z,x) = e 

kf(z) = bf'(z) + dv(2) = e^'”dz, 



X G M, z > 0. 


(6.13) 


The domain of operator is the closure in Cb(M) of C^-functions x G M g(^x) such that 
L^^'^g G Cb(M). The domain of A is the closure in Cb(M+) of C^-functions 2 ; G M+ t—)■ f(z) such that 
/'(0+) = 0 and A/ G Cb(IF+). Both and A generate unique Feller semi-groups. For the 

Feller semi-group is determined by the transition densities pt(x,x') from Eqn fl6.2p . For A, the Feller 
semi-group is determined by the transition densities 


{z, z') = 


e^bz _ X ' 7J- 


X 


/c 




^ 


s cos (s^;)-|-6sin (s^;) s cos -|-6sin (s^;') , z,z'>0 


(6.14) 


Cf. [I5], Eqns (28)-(29) (more precisely, a displayed equation between (28) and (29). 
In this example, for x G M and z > 0 we have: 


d z'^ ( 9 ^ 

R(f){z, x) = -x-^Hz, x) + -j-^Hz, x) 

^^2/^2 r, <9 ,, . . 192 


+6-^ 




(6.15) 


and 


dK(z,x) = e^^^-^'/^'dxdz. 


(6.16) 


The domain D(R) of operator R in fl6.15p is the closure of the set D‘’(R) from fl6.5p . Due to 
presence of function = [m{z,x)]~^, the existing general results do not guarantee that operator 

R generates a unique Feller semi-group in Cb(M+ x M). Nevertheless, Theorem 8 (I) holds, and k 
satishes the WIE with R. 

However, if we take X to be an interval I = (x^, x^) and Z = {z^, z^), with —00 < x^ < x^ < 00 and 
0 < z^ < z'^ < 00 , and assume that both and Z{t) are reflected at the endpoints x* and z^ then 
the situation changes. We still dehne R and k, via Eqns (16.15p and fl6.16p . for x G I, 2 ; G J, but the 


d d ■ 

domain D(R) involves the Neumann boundary conditions —(j){z,x^) = 0 and —(j){z'',x) = 0. Under 

ox ,^z 

these modihcations, operator R generates a unique Feller semi-group in Cb(J x I). Furthermore, by 


Proposition 9.2 in [3], P. 239, k is a genuine IM for this sem-group. 
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The corresponding MP is a pair {X(t), Z(t)) plus an auxiliary quadruple {Lx(t), Ux(t), Lz{t), Uz(t)) 
solving a system of SIDEs 


dX{t) = -X{t)dt + Z{t)dWi{t) + dLxit) - dUxit), X(0) = e ixi,X2) 

dZ{t) = —be^Xddt + dW 2 {t) + dLz{t) — dUz{t), Z{d) = ^ {zi, Z 2 ), 

l(X(s) = x,)dLxis) = Lxit), 1(X(5) = X 2 )dUx{s) = Uxit), 
jl l{Z{s) = zMLz{s) = Lz{t), jl liZis) = Z 2 )dUzis) = Uz{t), 


where Lx(t), Ux(t) are the local times spend by the process X(t) at points xi and X 2 and Lz{t), Uz(t) 
are local times spend by the process Z{t) at points Zi and Z 2 . 

Here and below, xo, zo are initial values and hPi(t), W 2 (t) are independent WPs in M. 

6.C. In this example we take ^ = M and deal with an OUP where the diffusion coefficient 

z follows its own OUP. Formally speaking, we set 

=-xg'{x) + ^g''{x), m(z, x) = 

.2 (6.17) 

Af{z) = -zf{z) + -r(z), dv{z) = e-^^dz, 

Here, the Feller semi-group generated by A is determined by the transition densities similar to fl6.2p : 


P 




IT 


exp 


jz' - ze-Y 

(1 — 


y G M. 


(6.18) 


Then, for 2 ;, a: G M, 


R 0 ( 2 ;, x) 


d z"^ 


+e' 




9 ^ 


1 

2^ 


0 ( 2 , x) 


(6.19) 


and 

dH,{z, x) = d;^. (6.20) 

The domain D(R) is the closure, in Cb(M x M), of C^-functions {z,x) G M x R (j){z,x) for which 
R0 G Cb(R X R). (The boundary condition in fl6.19p is omitted.) Again, the available general results 
do not allow us to conclude that there exists a unique Feller semi-group generated by R, but Theorem 
8 (I) holds true. 

However, as before, we can take X = (x^, x^) := I and Z = (z^ z^) := J where — 00 < < 

00 , —00 < z^ < z"^ < 00 and 0 ^ J, and use Eqns (16.191) and (16.201) for z G J, x G I, adding the 

d d ■ 

Neumann boundary conditions —(j)(z.x'') = 0 and —(j)(z^.x) = 0 in the dehnition of D(R). Then R 

ox oz 

generates a unique Feller semi-group in C(JI x I), and k is a genuine IM. 
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The corresponding diffusion MP is a pair {X{t), Z{t)) from the solution to SIDEs 


dX(t) = -X{t)dt + Z{t)dWi{t) + dLx{t) - dUx{t), X(0) = e {xi,X2), 

X^(t) X^(t) 

dZ{t) = -Z{t)e^dt + e^)dW 2 {t) + dLz{t) - dUz{t), Z(0) = e {zi, Z 2 ), 
jl 1(X(5) = xi)dLx{s) = Lx{t), l(X(s) = X 2 )dUx{s) = Ux{t), 
l(Z(s) = Zi)dLz(s) = Lz(t), jl liZ{s) = Z 2 )dUzis) = Uz{t). 


6.D. Finally, consider the case where the diffusion coefficient z follows a Cox-Ingersoll-Ross 
process on M_|_: see [2]. This example can be considered as a modification of the Heston model of a 
stochastic volatility; cf. [7]. Here we start with Z = M_|_ and generator A of the SE diffusion Z{t) of 
the form 

kf{z) = a{h - z)f{z) + |/"(^), ^v{z) = ^ > 0, (6.21) 

where a > 0, & > 0 are given parameters. The domain D(A) consists of C^-functions G M+ ^ f{z) 
such that A/ G Cb(M+) and - when a > 1/2 - the right derivative /'(0+) = 0. The Feller semigroup 
generated by A has the transition density 


r 1 (z '/ _\ 

p^*\z,z') = c exp (^ — cze~°‘^ — cz') I -j Iq \j2cVzz'e~^*j , z,z'>0, 


( 6 . 22 ) 


where c =- q = 2ab — 1 and /„ is the Bessel function of order q. (For a < 1/2, process Z(t) 

1 — ^ 

does not hit 0, whereas for a > 1/2 it hits 0 at infinitely many times that are indefinitely large.) 

The basic process is, as before, an OUP, with generator as in (16.Ih . (I6.13j) and (I6.17p . For 
X G M, 2 ; > 0, the combined generator is 


R(/(z, x) 


d . , z"^ 8“^ 


8{z,x) 


+e 




d 


a{b - z)—(j){z,x) + 


z 

2^ 


(l){z,x) 


(6.23) 


and we set 

dK{z,x) = 

The domain D(R) is the closure of D°(R) where 


D0(R) = |0 G 


X 


R0 G Ct 


dz 


(l){z,x) 


z=0+ 


R+ X M) and - for a > 1 /2 - also 
0 and lim R(/( 2 ;, x) = R(/(0+, x)|; 


(6.24) 


(6.25) 


cf. fl6.5p ). Again we can't derive that there is a unique Feller semi-group generated by R but Theorem 
8 (I) holds true. 
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Taking X = and Z = with —oo < < oo and Q < oo 

and introducing Neumann boundary conditions leads, as above, to a unique Feller semi-group for 
which K is a genuine IM. The coresponding diffusion MP is a pair {X{t), Z{t)) solving, together with 
(Lxit), Ux{t), Lz{t), Uz{t)), the system 

dX{t) = X{t)dt + Z{t)dWi{t) + dLx{t) - dUx{t), X(0) = G {xi,X 2 ) 

dZ{t) = a{b - Z{t))e^dt + ^/Z{{)e^)dW 2 {t) + dLz{t) - dUz{t), Z(0) = z^ e {zi, Z 2 ), 

£ 1(X(5) = x,)dLx{s) = Lx{t), £ l(X(s) = X 2 )dUx{s) = Ux{t), 

£ l{Z{s) = zMLz{s) = Lz{t), £ l(Z(s) = Z 2 )dUz{s) = Uz{t). 

A similar construction can be done if we consider the SE process Z{t) on an interval (0, r) instead 
of R+; cf. Section 6.1 in Ref. |15j . 

Remarks. 6.1. One of intriguing problems emerging from the above constuction (in its general 
form as well as in specihc examples) is to solve (and in fact, to pose in a correct manner) an inverse 
problem. (We can speak of a hidden Markov model.) Informally, we ask: is it possible to represent 
a given a random process X{t) as a projection (Z(f),X(f)) 1-4 X{t) of a combined MP {Z(t),X(t)) 
obtained by means of the construction, where component Z{t) describes a dynamics of the SE? E.g., 
in the context of sub-section 4.A: can one represent a birth-death process on Z_|_ as a result of a 
projection [z, n) & Z x Z_|_ 1 -^ n, from an MP on ^ x Z_|_ constructed in the above fashion? A similar 
question in the context of sub-sections 4.B is: given a process with continuous paths on (a domain D 
in) is it possible to represent it as a projection of an MP in a higher dimension, obtained by means 
of the above construction? In terms of Section 5, when it is possible to represent a given process 
with continuous paths on (a domain D in) as a projection of a diffusion in higher dimension? 

6.2. A feature of he construction presented in the paper is that the factor m{z, x) (and also a{z) 
in previous parts of the paper) enter both the generator R and IM /c in a mutually inverse fashion. 
It creates an interesting pattern: if m{z,x) is small then we have growing coefficients in R causing 
difficulties with proving the existence of the combined MP {Z{t),X{t)). But then we get an IM k 
which may be made an EPD (at least, at a formal level of the corresponding WIE). This suggests 
that for the models under consideration some new kind of existence theorems are possible, under 
weaker assumptions than in flS.lip . This may present an interesting direction for future development. 
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